<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.9.1"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>AIfES 2: aiopti_adam_default.h File Reference</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/x-mathjax-config">
  MathJax.Hub.Config({
    extensions: ["tex2jax.js"],
    jax: ["input/TeX","output/HTML-CSS"],
});
</script>
<script type="text/javascript" async="async" src="https://cdn.jsdelivr.net/npm/mathjax@2/MathJax.js"></script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
</head>
<body>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
 <tbody>
 <tr style="height: 56px;">
  <td id="projectlogo"><img alt="Logo" src="AIfES_logo_small.png"/></td>
  <td id="projectalign" style="padding-left: 0.5em;">
   <div id="projectname">AIfES 2
   &#160;<span id="projectnumber">2.0.0</span>
   </div>
  </td>
 </tr>
 </tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.9.1 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
var searchBox = new SearchBox("searchBox", "search",false,'Search','.html');
/* @license-end */
</script>
<script type="text/javascript" src="menudata.js"></script>
<script type="text/javascript" src="menu.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
$(function() {
  initMenu('',true,false,'search.php','Search');
  $(document).ready(function() { init_search(); });
});
/* @license-end */</script>
<div id="main-nav"></div>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
  <div id="nav-tree">
    <div id="nav-tree-contents">
      <div id="nav-sync" class="sync"></div>
    </div>
  </div>
  <div id="splitbar" style="-moz-user-select:none;" 
       class="ui-resizable-handle">
  </div>
</div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&amp;dn=gpl-2.0.txt GPL-v2 */
$(document).ready(function(){initNavTree('aiopti__adam__default_8h.html',''); initResizable(); });
/* @license-end */
</script>
<div id="doc-content">
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
     onmouseover="return searchBox.OnSearchSelectShow()"
     onmouseout="return searchBox.OnSearchSelectHide()"
     onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>

<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0" 
        name="MSearchResults" id="MSearchResults">
</iframe>
</div>

<div class="header">
  <div class="summary">
<a href="#nested-classes">Data Structures</a> &#124;
<a href="#typedef-members">Typedefs</a> &#124;
<a href="#func-members">Functions</a>  </div>
  <div class="headertitle">
<div class="title">aiopti_adam_default.h File Reference</div>  </div>
</div><!--header-->
<div class="contents">

<p>Default implementation of the <a class="el" href="aiopti__adam_8h.html">Adam optimizer </a>.  
<a href="#details">More...</a></p>

<p><a href="aiopti__adam__default_8h_source.html">Go to the source code of this file.</a></p>
<table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a name="nested-classes"></a>
Data Structures</h2></td></tr>
<tr class="memitem:"><td class="memItemLeft" align="right" valign="top">struct &#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="structaiopti__adam__f32.html">aiopti_adam_f32</a></td></tr>
<tr class="memdesc:"><td class="mdescLeft">&#160;</td><td class="mdescRight">Data-type specific <a class="el" href="aiopti__adam_8h.html">Adam optimizer </a> struct for <a class="el" href="aimath__f32_8h.html">F32 </a>.  <a href="structaiopti__adam__f32.html#details">More...</a><br /></td></tr>
<tr class="separator:"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table><table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a name="typedef-members"></a>
Typedefs</h2></td></tr>
<tr class="memitem:a13e16523a35fb20fc719337b5104e0b1"><td class="memItemLeft" align="right" valign="top"><a id="a13e16523a35fb20fc719337b5104e0b1"></a>
typedef struct <a class="el" href="structaiopti__adam__f32.html">aiopti_adam_f32</a>&#160;</td><td class="memItemRight" valign="bottom"><b>aiopti_adam_f32_t</b></td></tr>
<tr class="separator:a13e16523a35fb20fc719337b5104e0b1"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table><table class="memberdecls">
<tr class="heading"><td colspan="2"><h2 class="groupheader"><a name="func-members"></a>
Functions</h2></td></tr>
<tr class="memitem:a403603bdc0d77e2d1cbc3fd4cd37880a"><td class="memItemLeft" align="right" valign="top"><a class="el" href="structaiopti.html">aiopti_t</a> *&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="aiopti__adam__default_8h.html#a403603bdc0d77e2d1cbc3fd4cd37880a">aiopti_adam_f32_default</a> (<a class="el" href="structaiopti__adam__f32.html">aiopti_adam_f32_t</a> *opti)</td></tr>
<tr class="memdesc:a403603bdc0d77e2d1cbc3fd4cd37880a"><td class="mdescLeft">&#160;</td><td class="mdescRight">Initializes an <a class="el" href="aiopti__adam_8h.html">Adam optimizer </a> with the <a class="el" href="aimath__f32_8h.html">F32 </a> default implementation.  <a href="aiopti__adam__default_8h.html#a403603bdc0d77e2d1cbc3fd4cd37880a">More...</a><br /></td></tr>
<tr class="separator:a403603bdc0d77e2d1cbc3fd4cd37880a"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a0f5a44e9e363b8b8bbb1d3ad6b8e7658"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="aiopti__adam__default_8h.html#a0f5a44e9e363b8b8bbb1d3ad6b8e7658">aiopti_adam_f32_default_begin_step</a> (<a class="el" href="structaiopti.html">aiopti_t</a> *self)</td></tr>
<tr class="memdesc:a0f5a44e9e363b8b8bbb1d3ad6b8e7658"><td class="mdescLeft">&#160;</td><td class="mdescRight"><a class="el" href="aimath__f32_8h.html">F32 </a> default implementation of the <a class="el" href="structaiopti.html#a5b82d583ac0602253e07aaea992d4e3f" title="Called in the beginning of every model optimization step for parameter initialization.">aiopti.begin_step</a> function for ADAM  <a href="aiopti__adam__default_8h.html#a0f5a44e9e363b8b8bbb1d3ad6b8e7658">More...</a><br /></td></tr>
<tr class="separator:a0f5a44e9e363b8b8bbb1d3ad6b8e7658"><td class="memSeparator" colspan="2">&#160;</td></tr>
<tr class="memitem:a162ef93a1e3dc3c7fc3267dd6862c21b"><td class="memItemLeft" align="right" valign="top">void&#160;</td><td class="memItemRight" valign="bottom"><a class="el" href="aiopti__adam__default_8h.html#a162ef93a1e3dc3c7fc3267dd6862c21b">aiopti_adam_f32_default_end_step</a> (<a class="el" href="structaiopti.html">aiopti_t</a> *self)</td></tr>
<tr class="memdesc:a162ef93a1e3dc3c7fc3267dd6862c21b"><td class="mdescLeft">&#160;</td><td class="mdescRight"><a class="el" href="aimath__f32_8h.html">F32 </a> default implementation of the <a class="el" href="structaiopti.html#ae52778f5409e29c7f25cc4ee9ff66f55" title="Called in the end of every model optimization step.">aiopti.end_step</a> function for ADAM  <a href="aiopti__adam__default_8h.html#a162ef93a1e3dc3c7fc3267dd6862c21b">More...</a><br /></td></tr>
<tr class="separator:a162ef93a1e3dc3c7fc3267dd6862c21b"><td class="memSeparator" colspan="2">&#160;</td></tr>
</table>
<a name="details" id="details"></a><h2 class="groupheader">Detailed Description</h2>
<div class="textblock"><p>Default implementation of the <a class="el" href="aiopti__adam_8h.html">Adam optimizer </a>. </p>
<dl class="section version"><dt>Version</dt><dd>2.2.0 </dd></dl>
<dl class="section copyright"><dt>Copyright</dt><dd>Copyright (C) 2020-2023 Fraunhofer Institute for Microelectronic Circuits and Systems. All rights reserved.<br  />
<br  />
 AIfES is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.<br  />
<br  />
 This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.<br  />
<br  />
 You should have received a copy of the GNU Affero General Public License along with this program. If not, see <a href="https://www.gnu.org/licenses/">https://www.gnu.org/licenses/</a>.</dd></dl>
<p>Hardware independent implementations of the Adam optimizer in <a class="el" href="aimath__f32_8h.html">F32 </a>, <a class="el" href="aimath__q31_8h.html">Q31 </a> data-type. For more information about the Adam optimizer refer to <a class="el" href="aiopti__adam_8h.html" title="Base optimizer  implementation of the Adam optimizer">aiopti_adam.h</a>. </p>
</div><h2 class="groupheader">Function Documentation</h2>
<a id="a403603bdc0d77e2d1cbc3fd4cd37880a"></a>
<h2 class="memtitle"><span class="permalink"><a href="#a403603bdc0d77e2d1cbc3fd4cd37880a">&#9670;&nbsp;</a></span>aiopti_adam_f32_default()</h2>

<div class="memitem">
<div class="memproto">
      <table class="memname">
        <tr>
          <td class="memname"><a class="el" href="structaiopti.html">aiopti_t</a>* aiopti_adam_f32_default </td>
          <td>(</td>
          <td class="paramtype"><a class="el" href="structaiopti__adam__f32.html">aiopti_adam_f32_t</a> *&#160;</td>
          <td class="paramname"><em>opti</em></td><td>)</td>
          <td></td>
        </tr>
      </table>
</div><div class="memdoc">

<p>Initializes an <a class="el" href="aiopti__adam_8h.html">Adam optimizer </a> with the <a class="el" href="aimath__f32_8h.html">F32 </a> default implementation. </p>
<p><b>Example:</b> Create the optimizer structure:<br  />
In C: </p><div class="fragment"><div class="line"><a class="code" href="structaiopti__adam__f32.html">aiopti_adam_f32_t</a> adam_optimizer = {</div>
<div class="line">    .<a class="code" href="structaiopti__adam__f32.html#a7fa6b6264ed55a45f7af8b00eae05f59">learning_rate</a> = 0.01f,</div>
<div class="line"> </div>
<div class="line">    .beta1 = 0.9f,</div>
<div class="line">    .beta2 = 0.999f,</div>
<div class="line">    .eps = 1e-7f</div>
<div class="line">};</div>
<div class="ttc" id="astructaiopti__adam__f32_html"><div class="ttname"><a href="structaiopti__adam__f32.html">aiopti_adam_f32</a></div><div class="ttdoc">Data-type specific Adam optimizer  struct for F32 .</div><div class="ttdef"><b>Definition:</b> aiopti_adam_default.h:45</div></div>
<div class="ttc" id="astructaiopti__adam__f32_html_a7fa6b6264ed55a45f7af8b00eae05f59"><div class="ttname"><a href="structaiopti__adam__f32.html#a7fa6b6264ed55a45f7af8b00eae05f59">aiopti_adam_f32::learning_rate</a></div><div class="ttdeci">aiscalar_f32_t learning_rate</div><div class="ttdoc">Storage for aiopti.learning_rate scalar in F32.</div><div class="ttdef"><b>Definition:</b> aiopti_adam_default.h:54</div></div>
</div><!-- fragment --><p> In C, C++ and on Arduino: </p><div class="fragment"><div class="line"><a class="code" href="structaiopti__adam__f32.html">aiopti_adam_f32_t</a> adam_optimizer = AIOPTI_ADAM_F32(0.01f, 0.9f, 0.999f, 1e-7f);</div>
</div><!-- fragment --><p><b>Example:</b> Initialize the optimizer:<br  />
</p><div class="fragment"><div class="line"><a class="code" href="structaiopti.html">aiopti_t</a> *optimizer;</div>
<div class="line"> </div>
<div class="line">optimizer = <a class="code" href="aiopti__adam__default_8h.html#a403603bdc0d77e2d1cbc3fd4cd37880a">aiopti_adam_f32_default</a>(&amp;adam_optimizer);</div>
<div class="ttc" id="aaiopti__adam__default_8h_html_a403603bdc0d77e2d1cbc3fd4cd37880a"><div class="ttname"><a href="aiopti__adam__default_8h.html#a403603bdc0d77e2d1cbc3fd4cd37880a">aiopti_adam_f32_default</a></div><div class="ttdeci">aiopti_t * aiopti_adam_f32_default(aiopti_adam_f32_t *opti)</div><div class="ttdoc">Initializes an Adam optimizer  with the F32  default implementation.</div></div>
<div class="ttc" id="astructaiopti_html"><div class="ttname"><a href="structaiopti.html">aiopti</a></div><div class="ttdoc">AIfES optimizer interface.</div><div class="ttdef"><b>Definition:</b> aifes_core.h:438</div></div>
</div><!-- fragment --><dl class="params"><dt>Parameters</dt><dd>
  <table class="params">
    <tr><td class="paramname">*opti</td><td>The optimizer structure to initialize. </td></tr>
  </table>
  </dd>
</dl>
<dl class="section return"><dt>Returns</dt><dd>The (successfully) initialized optimizer structure. </dd></dl>

</div>
</div>
<a id="a0f5a44e9e363b8b8bbb1d3ad6b8e7658"></a>
<h2 class="memtitle"><span class="permalink"><a href="#a0f5a44e9e363b8b8bbb1d3ad6b8e7658">&#9670;&nbsp;</a></span>aiopti_adam_f32_default_begin_step()</h2>

<div class="memitem">
<div class="memproto">
      <table class="memname">
        <tr>
          <td class="memname">void aiopti_adam_f32_default_begin_step </td>
          <td>(</td>
          <td class="paramtype"><a class="el" href="structaiopti.html">aiopti_t</a> *&#160;</td>
          <td class="paramname"><em>self</em></td><td>)</td>
          <td></td>
        </tr>
      </table>
</div><div class="memdoc">

<p><a class="el" href="aimath__f32_8h.html">F32 </a> default implementation of the <a class="el" href="structaiopti.html#a5b82d583ac0602253e07aaea992d4e3f" title="Called in the beginning of every model optimization step for parameter initialization.">aiopti.begin_step</a> function for ADAM </p>
<p><em>Implementation of <a class="el" href="structaiopti.html#a5b82d583ac0602253e07aaea992d4e3f" title="Called in the beginning of every model optimization step for parameter initialization.">aiopti.begin_step</a>.</em></p>
<p>The ADAM optimizer needs some modification of the learning rate in every optimization step. This function deals with aiscalars and has to be implemented for every data-type individually.</p>
<p>The calculations are:<br  />
 </p><p class="formulaDsp">
\[ lr_t \leftarrow lr \cdot \frac{\sqrt{1 - \beta^t_2}} {(1 - \beta_1)^t} \]
</p>
<p>This is not primary time critical function, because it only deals with scalars -&gt; No special hardware implementation necessary (but possible).</p>
<dl class="params"><dt>Parameters</dt><dd>
  <table class="params">
    <tr><td class="paramname">*self</td><td>The optimizer structure </td></tr>
  </table>
  </dd>
</dl>

</div>
</div>
<a id="a162ef93a1e3dc3c7fc3267dd6862c21b"></a>
<h2 class="memtitle"><span class="permalink"><a href="#a162ef93a1e3dc3c7fc3267dd6862c21b">&#9670;&nbsp;</a></span>aiopti_adam_f32_default_end_step()</h2>

<div class="memitem">
<div class="memproto">
      <table class="memname">
        <tr>
          <td class="memname">void aiopti_adam_f32_default_end_step </td>
          <td>(</td>
          <td class="paramtype"><a class="el" href="structaiopti.html">aiopti_t</a> *&#160;</td>
          <td class="paramname"><em>self</em></td><td>)</td>
          <td></td>
        </tr>
      </table>
</div><div class="memdoc">

<p><a class="el" href="aimath__f32_8h.html">F32 </a> default implementation of the <a class="el" href="structaiopti.html#ae52778f5409e29c7f25cc4ee9ff66f55" title="Called in the end of every model optimization step.">aiopti.end_step</a> function for ADAM </p>
<p><em>Implementation of <a class="el" href="structaiopti.html#ae52778f5409e29c7f25cc4ee9ff66f55" title="Called in the end of every model optimization step.">aiopti.end_step</a>.</em></p>
<p>The ADAM optimizer needs some modification of the learning rate in every optimization step. This function deals with aiscalars and has to be implemented for every data-type individually.</p>
<p>The calculations are:<br  />
 </p><p class="formulaDsp">
\[ \beta^t_1 \leftarrow \beta^t_1 \cdot \beta_1 \]
</p>
<p class="formulaDsp">
\[ \beta^t_2 \leftarrow \beta^t_2 \cdot \beta_2 \]
</p>
<p>This is not primary time critical function, because it only deals with scalars -&gt; No special hardware implementation necessary (but possible).</p>
<dl class="params"><dt>Parameters</dt><dd>
  <table class="params">
    <tr><td class="paramname">*self</td><td>The optimizer structure </td></tr>
  </table>
  </dd>
</dl>

</div>
</div>
</div><!-- contents -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
  <ul>
    <li class="navelem"><a class="el" href="dir_d44c64559bbebec7f509842c48db8b23.html">include</a></li><li class="navelem"><a class="el" href="dir_1e5d3661ed79af157d57e64a38265d09.html">basic</a></li><li class="navelem"><a class="el" href="dir_6f3c54947e40ccd50db54894d07fbfc0.html">default</a></li><li class="navelem"><a class="el" href="dir_37dfedf0a42dfadb74a0ecf9fa01db6d.html">aiopti</a></li><li class="navelem"><a class="el" href="aiopti__adam__default_8h.html">aiopti_adam_default.h</a></li>
    <li class="footer">Generated by <a href="https://www.doxygen.org/index.html"><img class="footer" src="doxygen.svg" width="104" height="31" alt="doxygen"/></a> 1.9.1 </li>
  </ul>
</div>
</body>
</html>
